Dental and medical loupe system for lighting control, streaming, and augmented reality assisted procedures

ABSTRACT

In various examples, a dental and medical loupe system for lighting control, streaming, and augmented reality (AR) assisted procedures is provided. Facial recognition algorithms and/or inertial measurement unit (IMU) sensors may be used to determine when a light disposed on eye-glasses of a wearer should be activated. As a result, instead of requiring a medical practitioner to manually turn on and off the light source, facial recognition may be used to determine when the dentist is looking toward a face of a patient, and to turn on the light when a face is present and off when a face is not present. In addition, some examples leverage augmented reality (AR) functionality to overlay patient and procedure information on one or more lenses of eye-glasses worn by a practitioner to allow for an un-occluded view of the actual location where the procedure is being performed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.63/001,069, filed on Mar. 27, 2020, which is hereby incorporated byreference in its entirety.

BACKGROUND

Medical and dental professionals often wear eye-glasses, or other headworn devices, that may provide protection during their practice, provideincreased magnification during procedures, or provide supplemental lightto increase visibility. As an example, during dental practice, a dentistmay turn on and off a light mounted on their eye-glasses to provideincreased visibility within the oral cavity of a patient. These lightsare often powered by a battery pack, and the on/off switch is generallylocated on the same. However, because the dentist must manually turn thelight on and off using a switch, the light is often kept on forprolonged periods of time when not in use—such as when the dentist islooking away from the patient, has their eye-glasses removed and hangingfrom their neck, or otherwise forgets to turn off the light. As aresult, the battery is often quickly depleted, requiring the dentist touse another battery pack or, where not available, wait for the batterypack to charge. In addition, when turning the light switch on and off orchanging out the battery, the dentist is required to physically touchthe light switch with his or her hands, which may cause unnecessary andunsanitary exchanging of pathogens—an issue that violates cleanlinessand safety standards of dental practice.

In addition, although eye-glasses may be worn that include loupes tohelp increase magnification and thus visibility within the oral cavity,eye-glasses with loupes still do not help the dentist visualize regionsof the oral cavity that are hard to see, or to visualize bone, gum, orother structures within the oral cavity that are obscured or otherwiseconcealed by gum tissue, the tongue, skin, or other tissue. For example,when performing dental implants, root canals, or other dentalprocedures, dentists often rely on x-rays and surgical guides which notonly require extensive set-up (e.g., a first consultation for takingimpressions, a follow up days later for confirming the surgical guidefits, additional radiation exposure, etc.), but are often not asaccurate as desired. In addition, when performing these procedures usingsurgical guides, the dentist is often working on a region of the oralcavity that is obscured from view, and must rely on the surgical guidealone to execute the procedure. However, relying on the surgical guidealone without visual aid or confirmation may result in less-accurateprocedures that may not be as permanent or long-lasting, and may alsoresult in uneasiness or less confidence for the dentist performing theprocedure. Additionally, a pre-fabricated surgical guide solely relieson proper treatment planning and, even where necessary, doesn't allowfor any plan changes during surgery. As a result, the treatment plan maynot be carried out as effectively as desired or may be halted or put onhold until a new surgical guide can be fabricated.

This same thought process may apply to other medical fields, such asgeneral surgery, stent placements, pin, screw, or rod placements, jointreplacement surgeries, and/or other medical procedures. For example,many surgical procedures rely on cameras inserted into the body for aidin visualizing the procedure (e.g., during laparoscopy) or rely on otherinstruments such as continuous X-ray machines to visualize the procedure(e.g., during angioplasty). However, these medical devices are oftenexorbitantly expensive, are unavailable, or are limited in availabilitysuch that urgent procedures may be postponed while waiting for theiravailability. Similarly, in dental or other medical procedures, robotsmay be relied upon to aid in conducting the procedure. However, in eachof these procedure types—whether assisted by cameras, imagingtechnologies, robots, etc.—the medical practitioner relies on visualsthat are either obscured (e.g., such as during dental implants) ordistant from the patient (e.g., such as viewing a display screen showinga video from a camera, an X-ray, etc.). As such, the medicalpractitioner is unable to fully appreciate or clearly visualize themovement parts of the procedure with respect to the patient—therebyresulting in less than ideal operating conditions. Further, such as inangioplasty, the patient may undergo continuous radiation that may bedetrimental to the health of the patient (e.g., by absorbing continuousradiation).

SUMMARY

Embodiments of the present disclosure relate to a dental and medicalloupe system for lighting control, streaming, and augmented reality (AR)assisted procedures. Systems and methods are disclosed that use facialrecognition algorithms and/or inertial measurement unit (IMU) sensors todetermine when a light disposed on eye-glasses should be activated. Inaddition, systems and methods are disclosed that use AR functionality toaid in visualizing and guiding a medical practitioner through a medicalprocedure.

In contrast to conventional systems, such as those described above, oneor more cameras disposed on eye-glasses—that may or may not includeloupes—may be used to generate image data for use by a facialrecognition algorithm. For example, instead of requiring a dentist tomanually turn on and off the light source, facial recognition may beused to determine when the dentist is looking toward a face of apatient, and to turn on the light when a face is present and off when aface is not present. In addition, in some embodiments, and to furtherpreserve battery life, the detection of a face of a person may be usedto calibrate or zero-out IMU sensor(s), such that the IMU sensor(s) maythen be relied upon for turning on and off the light source. As aresult, the generation and/or transmission of image data may besuspended while the system relies on the IMU sensor(s). As a result, andin contrast to conventional systems, battery life may be preserved asthe light source may only be activated when the dentist is facing thepatient and infection control standards may be upheld as the dentist isnot required to manually turn on and off the light switch.

In further contrast to conventional systems that rely solely on internalcameras, imaging technologies, robots, and/or other tools to assist inmedical procedures, the systems and methods of the present disclosureuse AR functionality to overlay patient and procedure information on oneor more lenses of eye-glasses worn by a practitioner. For example, as anadditional or alternative source of visualization information, an ARoverlay may be generated and virtually projected in the field of view ofthe practitioner to aid the practitioner in performing a procedure. In adental implant, for example, a patient's CT scan of one or more teethmay be virtually overlaid on the patient—e.g., on the patient's cheekusing one or more markers or guides for reference—at a location thatcorresponds to the actual location of the one or more teeth. As such,the practitioner may be able to look toward the actual location of wherethe procedure is being performed, without visual occlusion, therebyincreasing the accuracy of the procedure and increasing confidence ofthe practitioner with respect to the same—in a senses, giving thepractitioner x-ray vision.

BRIEF DESCRIPTION OF THE DRAWINGS

The present systems and methods for lighting control, streaming, andaugmented reality (AR) assisted procedures are described in detail belowwith reference to the attached drawing figures, wherein:

FIG. 1A is an example illustration of a loupe system in use during adental procedure, in accordance with some embodiments of the presentdisclosure;

FIG. 1B is an example illustration from the perspective of a user of anAR-assisted loupe system, in accordance with some embodiments of thepresent disclosure;

FIG. 1C is an example illustration of providing notifications of patientinformation to a user of an AR-assisted loupe system, in accordance withsome embodiments of the present disclosure;

FIG. 1D is another example illustration from a perspective of a user ofan AR-assisted loupe system, in accordance with some embodiments of thepresent disclosure;

FIG. 1E is a block diagram of an example loupe system for lightingcontrol, streaming, and augmented reality (AR) assisted procedures, inaccordance with some embodiments of the present disclosure;

FIG. 2A is an example illustration of eye-glasses electrically coupledto a power component in a loupe system, in accordance with someembodiments of the present disclosure;

FIG. 2B is an example illustration of a housing associated with ahead-worn device within a loupe system, in accordance with someembodiments of the present disclosure;

FIGS. 3A-3G are example illustrations of graphical user interfaces(GUIs) for streaming or viewing medical procedures and/or viewingpatient information within one or more applications, in accordance withsome embodiments of the present disclosure;

FIG. 4 is a flow diagram for an example method of using facialrecognition to activate or deactivate a light source of eye-glasses in aloupe system, in accordance with some embodiments of the presentdisclosure;

FIG. 5 is a flow diagram for an example method of streaming a medicalprocedure captured using a head-mounted camera to one or more remoteclient devices, in accordance with some embodiments of the presentdisclosure;

FIG. 6 is a flow diagram for an example method of using AR functionalityto provide virtual visualizations to assist in medical procedures, inaccordance with some embodiments of the present disclosure; and

FIG. 7 is a block diagram of an example computing device suitable foruse in implementing some embodiments of the present disclosure.

DETAILED DESCRIPTION

Systems and methods are disclosed related to a dental and medical loupesystem for lighting control, streaming, and augmented reality (AR)assisted procedures. The disclosure is not intended to be limited to anyparticular medical field or practice, and may correspond to dentistry,general medical diagnosis and practice, surgical specialties, and/or anytype of professional practice that may benefit from facial recognitionbased light source control and/or AR assisted visualizations. As such,even though the disclosure herein may be directed primarily to thedental or medical field, this is not intended to be limiting, and thesystems and methods described herein may be used within additional oralternative practice fields or with different technologies altogetherwithout departing from the scope of the present disclosure.

With reference to FIGS. 1A-1E, FIGS. 1A-1E illustrate an example loupesystem(s) 102 for lighting control, streaming, and augmented realityassisted procedures, in accordance with some embodiments of the presentdisclosure. It should be understood that this and other arrangementsdescribed herein are set forth only as examples. Other arrangements andelements (e.g., machines, interfaces, functions, orders, groupings offunctions, etc.) may be used in addition to or instead of those shown,and some elements may be omitted altogether. Further, many of theelements described herein are functional entities that may beimplemented as discrete or distributed components or in conjunction withother components, and in any suitable combination and location. Variousfunctions described herein as being performed by entities may be carriedout by hardware, firmware, and/or software. For instance, variousfunctions may be carried out by a processor executing instructionsstored in memory. In some non-limiting embodiments, one or more of thecomponents of the loupe system(s) 102 may implement one or more of thecomponents of example computing device 700 of FIG. 7, described in moredetail herein. However, additional or alternative components other thanthose illustrated in FIG. 7 may be implemented without departing fromthe scope of the present disclosure. In addition, although referred toherein as a loupe system(s) 102, in some embodiments, no loupes may beimplemented or included (e.g., eye-glasses or other head worn devicesmay be used without the use of loupes for magnification).

With reference to FIG. 1A, in some embodiments, the loupe system(s) 102may be used for activating and/or deactivating a light(s) 124 (e.g.,LED, halogen, CFL, etc.) disposed on a head-worn device 130—e.g.,eye-glasses, a hat, a helmet, a visor, goggles, dental loupes, operatingglasses, etc. For example, as described herein, conventional systemssuffer from shortened battery life as a result of lights being left on,kept on when not in use, etc. In addition, conventional systems do notaccount for where the light is shining, such as in the eyes of a patientor assistant, and further require a user to physically turn on and offthe light source using a button or switch—thereby causing transfer ofgerms and bacteria that may result in unaccounted for yet unsanitaryoperating or clinical conditions. As such, to account for each of thesedrawbacks of conventional systems, the present loupe system providesfunctionality for automatically turning on and/or off the light(s) 124using one or more cameras 116, one or more IMUs 112—or sensorsthereof—and/or one or more input/output (I/O) devices 126 (e.g.,microphones, speakers, etc.).

For example, in some embodiments, the IMU(s) 112 may be used to turn onand/or off the light(s) 124. In such examples, the IMU(s) 112 may detectcertain orientations, movements, or positions with respect to thehead-worn device 130 (e.g., positions that indicate the user 160 isfacing a person 168, or is otherwise in a position programmed tocorrespond to a light on position) that indicate that the light shouldbe turned on, and similarly may detect certain orientations, movements,or positions that indicate the light should be turned off. For example,where a quick movement is detected, the light may be turned off, orwhere a lack of movement is detected for a threshold amount of time, thelight may be turned on. In other examples, the IMU(s) 112 may becalibrated such that a zero position represents an orientation,movement, and/or position of the head-worn device 130 that is a light onposition. As such, this position—and an operating degree of freedomtherefrom—may be monitored to determine when the light should be on andwhen the light should be turned off. As a result, the light(s) 124 maybe turned on and off depending on a variety of factors and based on datafrom the IMU(s) 112.

In at least one embodiment, a combination of the camera(s) 116 and theIMU(s) 112 may be used. For example, the image data from the camera(s)116 may be analyzed to determine that a person 168—or a featurethereof—is present, and this confirmation may be used to calibrate orzero out the IMU(s) 112. As such, once the person 168 is detected, thecamera(s) 116 may stop generating the image data and/or transmission ofthe image data to the local client device(s) 104 (where applicable) maybe ceased, and the IMU(s) 112 may be used as the determiner of when toturn on and/or off the light. For example, where the IMU(s) 112 arezeroed out after a determination that a face of a person 168 is present,data from the IMU(s) 112 may be analyzed to determine whether certainmovements, positions, and/or orientations are met that indicate thelight should be turned off. Once a movement, position, and/ororientation is met that indicates the light should be turned off, thedata from the IMU(s) 112 indicating that a subsequent movement,position, and/or orientation puts the head-worn device 130 back within athreshold of the zeroed out position may cause the light to be activatedagain. As such, once a person 168 or feature thereof is detected, andthe IMU(s) 112 are zeroed or otherwise calibrated, the IMU(s) 112 may beused exclusively during the particular session and/or until a thresholdamount of deviation is detected that prompts another determination ofthe presence of the person 168. This may help to further increasebattery life. In some examples, the camera(s) 116 may be reactivatedperiodically—e.g., at an interval—to capture updated image data that maybe analyzed to determine a presence of a person or feature thereof inorder to recalibrate or zero-out the IMU(s) 112.

As an example, where a dentist is using the light(s) 124 to illuminatethe mouth of the person 168, the movement of the dentist's head (or thehead-worn device 130) upward a certain degree (e.g., 10 degrees, 15degrees, etc.) from the zeroed position may align with the eyes of theperson 168. In some embodiments, the degree of movement that correspondsto eyes of the person 168 may be individualized for each patient. Forexample, facial recognition, computer vision, machine learning, or othertechniques may be leveraged to develop a zero or calibration point forthe area of focus for a particular patient and/or for the degree ofmovement from a zero or calibration point at which the light should beturned off. As a result, the IMU(s) 112 may be calibrated for eachindividual function thereby increasing accuracy and scalability ofimplementation of the loupe system 102 across a wider range of patientsor persons. As such, when a movement above this threshold in the upwarddirection is detected using the data from the IMU(s) 112, the light maybe switched off, and when the dentist moves back toward the mouth of theperson 168—and thus within the threshold—the light may be reactivated.Similarly, when looking downward away from the mouth, a larger thresholdmay be used (e.g., 20 degrees, 40 degrees, etc.) because the dentist mayneed to look for tools, cloths, napkins, or other items, as well as lookat status information of one or more machines. In addition, because amovement downward likely will not result in shining the light(s) 124 inthe eyes of the person 168, the downward threshold may be greater thanthe upward threshold. Other thresholds may be used for the left andright rotation or movement, such that a movement threshold in anydirection may be put in place, and monitored in view of the data fromthe IMU(s) 112.

In some embodiments, the camera(s) 116 alone may be used to detect aperson or a feature thereof. For example, image data generated by thecamera(s) 116 may be analyzed by the loupe system(s) 102, the localclient device(s) 104 (e.g., a computer, laptop, phone, tablet, etc.),and/or server(s) 106 to determine when a person or feature thereof ispresent, and the light(s) 124 may be activated and deactivated based onthis determination. As such, the image data captured by the camera(s)may be analyzed using one or more computer vision, facial recognition,and/or other detection algorithms to determine the presence of a person168, a face of a person 168, another feature of a person 168, and/or thelike. In any embodiment where image data is analyzed, the facialrecognition or other recognition may be person specific. For example,the particular patient and corresponding stored facial recognition datamay be determined such that the activation or deactivation of thelight(s) 124 is only for the particular person. As such, if the face ofa nurse, assistant, friend or family member, or other person other thanthe patient is detected, this detection may not cause activation of thelight(s) 124. However, detecting the face of another person other thanthe patient may prompt deactivation of the light(s) 124.

In some examples, the image data may be continually generated and/oranalyzed (e.g., a 30 frames per second (FPS), 60 FPS, etc.), while inother examples, the image data may be generated periodically (e.g., 1FPS, 2 FPS, etc.) such as to preserve battery life. Depending on theparticular procedure, certain indicators may be identified or detectedusing the image data to determine when the light(s) 124 should be turnedon and off. For example, during general dentistry, when a mouth of aperson is at a particular location within the image data (e.g.,centered, within a central region, within a portion of the imageilluminated by light from the light(s) 124, etc.) the light may beactivated. When the mouth is at a different location, or the light fromthe light(s) 124 is determined to be at or near the eyes of the person168, the light may be deactivated. These determinations may bedetermined using one or markers on the person, guides on the person, orfeatures of the person as reference. For example, where a feature ormarker is identified at a particular location in an image represented bythe image data (e.g., above a certain pixel row), the light may bedeactivated as this location may indicate that the light(s) 124 islikely shining in the eyes of the person. Similarly, when a person isnot detected at all—of a face or other features are not detected—thelight may be deactivated. As such, as a user 160 looks away from theperson 168, looks into the person's eyes, looks down at the floor, at acomputer screen, reaches for a tool, etc., the light(s) 124 may bedeactivated—or may be dimmed, such that the amount of light is not adisturbance to the patient or others. These determination may be madeusing one or more computer vision, facial detection, or other detectionalgorithms, and may aid in extending battery life for the loupesystem(s) 102.

In some non-limiting embodiments, additional cues or information may beused to activate or deactivate the light(s) 124. For example, a heatsensor(s) 180, or other sensor type, may be used to detect that thehead-worn device 130 is no longer being worn by the user 160 (e.g., hasbeen removed, is put into a non-operating position (e.g., on top ofuser's head, around user's neck, etc.), and/or is otherwise no longerbeing used. For example, the heat sensor(s) 180 may be on the nose padof eye-glasses, on the temples along the ears of eye-glasses, inside ahelmet in contact with the user's head or forehead, and/or otherwiselocated on the head-worn device 130 and capable of detecting changes intemperature that indicate the head-worn device 130 is no longer inoperating position. In some embodiments, a threshold period of time maybe used to determine when to turn on or off the light(s) 124, such as athreshold amount of time (e.g., 3 seconds, 5 seconds, 10 seconds, etc.)where the heat sensor(s) 180 indicates the head-worn device 130 is notin operating position. In addition, a microphone(s), a button (e.g.,button 202 of FIG. 2A), a switch, a touch-sensitive surface, and/oranother I/O device(s) 126 on the head-worn device 130, a power component164, and/or another component of the loupe system(s) 102 may be used toturn on and/or turn off the light. For example, the user 160 may providevoice commands to a microphone(s) to turn on or off the light(s) 124and/or to control the functionality of both photo and video (e.g.,“start recording”, “capture image”, “stop recording”, etc.), may turn onor off the light with a button or switch, may press or touch atouch-sensitive or heat-sensitive surface (e.g., on the rims or templesof eye-glasses). In addition, in some embodiments, such as where apatient's eyes may be fully or partially occluded (e.g., to protectiveeye-wear, head-wear, etc.), markers may be placed on or disposed on(e.g., during manufacture) a patient's protective eye-wear, head-wear,and/or the like, and the markers may be leveraged by the loupe system102 to perform various functions—e.g., to turn on or off the light, toturn on or off the camera, etc. For example, when a user of the loupesystem 102 raises his or head up towards the patient's eyes, imageprocessing techniques, machine learning models, and/or the like may beused to identify the markers and trigger the associated operations orcommands.

In addition to controlling activation or deactivation of the light(s)124, the light(s) 124 may be controlled based on the image data and/orthe IMU(s) 112. For example, one or more motors 183 may be used tomaneuver or manipulate the light(s) 124 with respect to the head-worndevice 130. As such, as the user 160 moves his or her head, the light(s)124 may be adjusted to maintain light coverage within a certain regionor area of the person. The manipulation of the light(s) 124 using themotor(s) 182 may be determined from the image data, such as byidentifying the light in an image(s) and/or by identifying markers,guides, or features of the person in the image that may be used todetermine where the light is shining. For example, intrinsic and/orextrinsic parameters of the light(s) 124 and the camera(s) 116 capturingthe image data may be known, and the relative locations, positions,orientations, and/or other parameters of the light(s) 124 and thecamera(s) 116 may be determined from this information. As such, adetermination of how to adjust or manipulate the light(s) 124 as theuser 160 moves his or her head may be determined using this informationsuch that the light(s) 124—when physically capable of doing so—isshining at a particular region or within a threshold region. In suchexamples, where the light(s) 124—based on a position, orientation, orother aspect of the head-worn device 130—is unable to shine at thedesire region or within the threshold region, the light may bedeactivated.

With reference to FIG. 1B, in some embodiments, the loupe system(s) 102may be used for augmented reality applications. For example, an ARsystem 120 may be implemented using the loupe system(s) 102, the localclient device(s) 104, and/or the server(s) 106. As such, the processingof image data generated by the camera(s) 116, the processing of medicalimaging data generated by one or more medical imaging devices, and thegenerating of AR-content using the image data and the medical image datamay each be performed by one or more of the loupe system(s) 102, thelocal client device(s) 104, or the server(s) 106. Presentation of theAR-content—e.g., projection of the AR-content on a lens(es) of thehead-worn device 130—may be performed by the loupe system(s) 102, andspecifically by an AR projection device disposed on or otherwiseassociated with the head-worn device 130, in non-limiting embodiments.

As described herein, the medical imaging data may be generated by one ormore imaging modality from the array of imaging modalities, such as butnot limited to X-rays, computed tomography (CT), magnetic resonanceimaging (MRI), ultrasounds, or nuclear imaging.

For example, although not illustrated in FIG. 1B, the head-worn device130—which is illustrated as eye-glasses in FIG. 1B—may include at leasta portion of the AR system 120 disposed thereon. The AR system 120 mayinclude one or more projection devices or components for projectingAR-content on one or more lenses 162 (e.g., lens 162A and 162B) of thehead-worn device 130. The AR-content may be generated and projected ordisplayed using any technique, such as but not limited to thosedescribed herein. For example, the camera(s) 116 disposed on thehead-worn device 130 may be used to generate image data of the person168 in real-time or near real-time. In some embodiments, the person 168may have one or more markers 172 (or guides) attached thereto. Althoughonly one marker 172 is illustrated, this is not intended to be limiting,and any number of markers may be used. For example, where computervision triangulation (or reconstruction), two-dimensional (2D) mapping,three-dimensional (3D) mapping, or another tracking technique is used,the head-worn device 130 may include two or more cameras 116 (e.g.,camera 116A, 116B and 116C as illustrated in FIG. 1A) and two or moremarkers 172 and/or features of the person. The “C” on the marker 172 mayindicate center, and additional markers—such as “L” for left and “R” forright—may also be attached on a left side and a right side of the “C”marker 172 at some location on the person 168, respectively. Asdescribed in more detail herein, the markers 172 may be used foraccurate alignment of the AR-content projection with respect to theperson 168 to provide a more clear visualization of patient data (e.g.,medical imaging data) related to the person 168 for the user 160 of thehead-worn device 130. In addition to, or alternatively from, using themarkers 172, other alignment techniques may be used. For example,features of the person 168—such as eyes, ears, nose, mouth, lips, jaw,etc.—may be used to determine positioning for projection of theAR-content.

In any embodiment, the locations of the markers 172 and/or the featuresof the person 168 may be determined using the image data generated bythe camera(s) 116, and this information may be used to align medicalimaging data with the markers 172 and/or features from the image data togenerate AR-content that is to scale, as well as to project theAR-content at a location that aligns accurately with a real-worldlocation of the feature of the person 168 virtually represented in theAR-content. For example, with respect to FIG. 1B, X-ray data may becorrelated with the image data to determine AR-content 170 that includesteeth of the person 168. The scale of the AR-content 170 may bedetermined using one or more markers 172 and/or features of the person168, and the location of the projection of the AR-content—e.g., overlaidon a cheek of the person 168 corresponding to an actual location of theteeth of the person behind the cheek wall—may also be determined usingthe markers 172, the features of a patient's face, the features and/oranatomy of the patient's oral cavity, etc. For example, because theperson may have been wearing the markers 172—e.g., radiographicmarkers—during the generating of the medical imaging data using themedical imaging device, and may still be wearing the markers 172 duringthe AR projection, the alignment of the AR projection 170 with theperson 168 may be accurately determined with medical precision. As aresult, when the user 160 of the head-worn device 130 is performing aprocedure on the person 168 that may have conventionally be a blindprocedure, or at least partially blind procedure, the user 160 may beable to visualize the location of features (e.g., teeth) of the personbased on the AR-content. In addition, by using this visualizationtechnique, the degree of comfort for the person 168 may be increased.

For example, with respect to a dental procedure, the person 168 may notbe forced to open their mouth as wide, or may be able to have smallersized mouth props, because the user 160 (e.g., dentist or oralsurgeon)—due to the accurate and additional visualization from theAR-content 170—may not need as much of an opening in the oral cavity toeffectively perform the procedure. For an implant surgery, adetermination of the depth, position, and angulation may be made, andthis determination may aid the generation of the AR-content for displayor projection on a lens of the head-worn device 130. For example, anindicator may be included in the AR-content such that a dentist or oralsurgeon may place the implant in the ideal location. As such, theAR-content that may be generated from X-ray data and image datagenerated by the camera(s) 116 may be supplemented with indicators orother information to aid in performing one or more procedures. In suchan example, the location, orientation, or other information of the toolbeing used to perform the procedure may be tracked, and thus known, andthis information may be used to update the visualization to provide avisual cue that the depth has been reached. In addition to a visual cue,an audible, tactile, haptic, or other cue type may be generated. In someexamples, a visual representation of at least a portion of the tool(s)being used for a procedure may be included in the AR-content andprojected in the visualization. This may be useful in many procedures orsurgeries, such as where the tools, components, or other devices areobscured either by skin, bone, organs, and/or other obstacles (e.g.,hands or other body parts of the surgeon, the surgeon's helpers, etc.).For example, with an implant for a molar, being able to visualize thelocation, orientation, and depth of the drill within the visualizationfrom the AR-content may increase accuracy, awareness, and confidenceduring the procedure.

Similar benefits may be realized during tooth implants, bone grafts,and/or the like. For example, during a tooth implant, tissue flapopening may be required to visualize the bone underlying the tooth toincrease accuracy during the procedure. However, this procedure is notonly more invasive, it also requires longer recovery periods andincreases risk of infection. Some approaches use surgical guides createdfrom molds, and then use a tissue punch to access the bone forimplanting, but this technique also requires that the mold is accuratelycreated, and leaves no room for adjustment or visualization during theactual procedure. In addition, the creating of the mold requires a firstvisit, and then the mold must be fitted at a subsequent visit to confirmthat the mold fits. If the mold does not fit, a new one must be created,thus potentially further prolonging the already extended period forrepairing what may be an uncomfortable or painful intraoral issue.However, using the systems and methods described herein may allow forvisualizations of the bone structure using AR-content generated usingmedical imaging data, image data from the camera(s) 116, markers,features of the person, and/or other information. As such, as thedentist or oral surgeon performs a tooth implant, the location of thebone (e.g., to ensure a central-placed implant), the location of thedrill, and/or the location of surrounding teeth may be included in thevisualization to help guide the procedure. In some embodiments, surgicalguides may still be used, or robots may be used, and the AR-content maysupplement the use of the surgical guides, robots, and/or other devices.For example, using the visualization from the AR-content in addition tothe surgical guide (which may also be virtually represented in theAR-content) may allow for more confident and accurate procedures. Asanother example, the use of a robot for a tooth implant (or anothermedical procedure) may be supplemented with the AR-content to help thesurgeon or doctor visualize the robotic components as the procedure istaking place—thereby increasing accuracy and confidence while alsopreventing previously unidentifiable issues.

As another example, and with respect to bone fracture repair, markersand/or features of the person may be captured in the medical imagingdata as well as the image data from the camera(s) 116, and thevisualization from the AR-content created as a result may help guidepositioning of screws, pins, rods, plates, and/or other repaircomponents. This additional visualization from AR-content may allow forless invasive surgery—and less scarring—as a result of the externalvirtual visualization allowing for reduced internal real-worldvisualization (e.g., incisions or other openings of the skin may bereduced). Additionally, AR-content generated within the loupe system 102may aid in the closure of a fracture by helping the clinician place thebones in the correct position prior to the fabrication of a cast. Thismay help avoid surgery in certain cases by allowing for ideal fracturereduction without surgery.

As a further example, with respect to heart stent placement, the stentmay be inserted through a femoral artery in the leg and tracked upthrough to the heart. Using the loupe system(s) 102, described herein,an AR visualization may be generated to help the user (e.g., surgeon)track the stent as the stent—and associated components/tools—make theirway up through the femoral artery. Traditionally, live X-rays or otherimaging techniques are used to capture continuous images of the stent asthe stent moves through the body, and the surgeon may view the liveX-rays or other images to guide the stent. However, this traditionalapproach takes the surgeon's eyes off of the patient, requires that thesurgeon use a visualization from the X-rays to guide the stent, and alsogreatly increases radiation exposure to the patient, surgeon, and allpersonnel inside the operating room. This dissociation of the surgeon'sfocus from the actual placement increases the complexity of theprocedure, and may result in more complications. Using the systemdescribed herein, AR-content may be used to replace or supplementtraditional techniques such that the surgeon may visualize the locationof the stent and/or surrounding arteries and organs as the AR-content isoverlaid on the patient during the procedure. For example, theAR-content may serve as “X-ray vision” for the surgeon and allow thesurgeon to look at the actual location of the stent within the patientwhile looking at the patient—and not at a screen or other displayshowing X-ray images. In this example, the live X-rays may capture theperson, the stent, one or more features of the person, and/or one ormore markers on the person, and this information may be used incombination with image data from one or more camera(s) 116 to generatethe AR-content. The AR-content may then be projected onto the lens ofthe head-worn device 102 such that a virtual visualization of the liveX-rays is overlaid onto the person in semi-virtual space.

With respect to FIG. 1C, additional AR-content may be generated fordisplay to the user of the head-worn device 130. For example, forpatient records and/or for insurance purposes, images of cavities, toothimplants, missing teeth, misaligned teeth, etc. may be captured. Inconventional approaches, intra-oral cameras may be used to capture thisinformation, or an external camera may be used to capture thisinformation. However, the capturing of this data required that the toothto be captured was determined by a dentist or assistant, the image wascaptured, and the image was then analyzed by a person to determine thatthe tooth was actually captured. Using this technique, images are oftennot captured or are not clear enough to satisfy insurance standards. Inaddition, the use of intra-oral cameras is often unsanitary as the userof the camera must manipulate the camera within the oral cavity, whichmay lead to transfer of bacteria or germs to the practitioner. As such,in some embodiments of the present disclosure, AR-content (e.g.,AR-content 174A, 174B, 176A, and 176B of FIG. 1C) may be generated tofacilitate the accurate and complete capture of the necessaryinformation for patient records and/or insurance purposes while alsoincreasing compliance with sanitary standards.

For example, a medical application(s) 134 on the local client device(s)104 may include an identifier of what information needs to bememorialized via images. This determination may be made by a person(e.g., doctor, assistant, etc.) and/or may be made by an analysis ofmedical imaging data and/or image data. For example, the medicalapplication(s) 134 (and/or one or more applications on the server(s)106) may analyze patient medical imaging data and/or image data todetermine where cavities exist (e.g., by analyzing bone densityinformation), where teeth need to be replaced, where gaps exists, whereteeth are misaligned, etc., and may populate this information in themedical application(s) 134. The information may then be used to aid theuser 160 of the head-worn device 130 in capturing the necessary imagedata while with the person 168 (e.g., a patient). Textual informationmay be generated within the AR-content to indicate to the user 160 whatneeds to be captured, adjustments that need to be made to capture asuitable image (e.g., turn to left, right, up, or down, open up mouth ofpatient wider, etc.), and that a suitable image was captured. Thedetermination that a suitable image was captured may be made by themedical application(s) 134 (executing on the local client device(s) 104and/or the server(s) 106) by analyzing the image data generated by thecamera(s) 116—e.g., using a computer vision algorithm or anotheralgorithm. The location and scale of the projection of the AR-contentmay be determined, similar to as described herein, using markers,guides, features of the person, and/or other supplemental information.

For example, with respect to the upper portion of FIG. 1C, theAR-content 174A may provide an indication projected on a lens(es) 162 ofthe head-worn device 130 reciting “Need image of pit cavity on frontsurface of tooth 3.” In addition to the textual indication, theAR-content 176A may provide a visual indication as an overlay of X-raydata (or other medical imaging data) on the person 168 with a visualindicator on the tooth to be captured. Once it has been determined thata suitable image of the tooth has been accurately captured, theAR-content 174B may provide an indication projected on a lens(es) 162 ofthe head-worn device 130 reciting “Image of pit cavity on front surfaceof tooth 3 captured.” In addition to the textual indication, theAR-content 176B may provide a visual indication as an overlay of X-raydata (or other medical imaging data) on the person 168 with a visualindicator on the tooth that the image was captured (e.g., a check markin the example illustration of FIG. 1C). As a result of this process,each of the images that need to be captured may be presented to the user160 without requiring the user to manually determine and confirm each ofthe images has been captured. This not only expedites the process ofcapturing the proper images, but also increases the accuracy andthoroughness of patient records as well as insurance claims. In someembodiments, the images that are captured may be uploaded and saved foraccess by the person 168 (e.g., the patient) via his or her remoteclient device(s) 108 within a client application(s) 150. For example,the person 168 may be able to access the images and insuranceinformation, as well as other information via patient portal—asdescribed in more detail herein.

In some non-limiting embodiments, the image data captured over time fora particular person 168 (e.g., patient) may be analyzed to trackchanges. For example, with respect to a dentist or orthodontist, theimages captures using the camera(s) 116 may be analyzed over time todetect movement or changes in condition of teeth (e.g., presence ofcavities, color changes, etc.). Not only can this information be helpfulfor determining a progression of the teeth (either positive or negative)over time, this information may be used to aid in generating AR-contentfor the user 160 (e.g., the dentist, orthodontist, oral surgeon, etc.).For example, the current location of a tooth (or teeth) may be used—inaddition to one or more features of the person and/or markers orguides—to determine a location to virtually overlay a prior location ofthe tooth (or teeth). This way, the user 160 may visualize the movementby comparing the current location to the prior location(s) capturedduring prior visits as captured in the image data. In some embodiments,the movement may be analyzed by a medical application(s) 134 todetermine if the movement is positive (e.g., the braces have moved theteeth toward a final, goal location) or negative (e.g., the patient'steeth have moved more out of alignment causing potential jaw oralignment issues), thereby enabling the comparison of actual movement toexpected movement according to a treatment plan. This information may bememorialized for later review by the patient, the doctor, the dentist,etc., or may be used to aid in generation of AR-content. For example, anindicator may be overlaid on the tooth (or teeth) of the patient withinthe AR-content to indicate to the user 160 that the particular tooth (orteeth) has moved in a positive or negative direction. This informationmay help the user 160 in determining a proper course of action, such asto adjust braces, recommend braces, remove teeth, and/or to performanother procedure type.

With reference to FIG. 1D, FIG. 1D is another example illustration froma perspective of a user of an AR-assisted loupe system, in accordancewith some embodiments of the present disclosure. For example, the loupesystem 102 may be used to generate AR-content (e.g., AR-content 190A and190B) to aid a user of the head-worn device 130 in performing one ormore procedures. With respect to FIG. 1D, the AR-content 190 may begenerated for tumor removals or visualization of tumors when examining aperson 168. For example, medical imaging data may represent one or moretumors, and when extracting the tumor(s), the AR-content 190 may providea virtual visualization of a tumor 194 and its location with respect tothe person or patient. As described herein, various techniques may beused to leverage markers 192 (e.g., 192A-192D) for accurately localizingthe AR-content 190 with respect to the person 168. This information mayaid the surgeon in determining where to start the incision (e.g., atpoint “A”), as well as how large the incision should be (e.g., to end atpoint “B”). In some examples, the incision may be determined as part ofthe AR-content generation, such that a virtual representation of theincision is included in the AR-content 190B and virtually overlaid onthe person or patient 168 at the location of the incision. As a result,the surgeon or user of the head-worn device 130 may create the incisionby following the incision from the virtual representation in theAR-content. In some embodiments, the AR-content may help guide thesurgeon through the different layers of tissue while performing asurgery. For example, the AR-content 190 may provide visual indicatorscorresponding to various vital structures—e.g., nerves and bloodvessels—that are known to exist in each layer of tissue. As such, theloupe system 102 may track—e.g., using image processing techniques, suchas those described herein—the procedure layer by layer and automaticallygenerate AR-content 190 to aid the user. In such an example, once afirst layer is cut through, a notification may appear as AR-content 190on a lens 162 that indicates what to be on the look for (e.g., “avoidnerve X”) in the next and currently visible and/or AR-content 190indicating a known (e.g., from medical imaging data and localized viatriangulation or other techniques) or estimated location of a vitalstructure may be generated and overlaid on the person or patient 168.This process may greatly aid in the avoidance of vital structures, e.g.,nerves and blood vessels.

Additionally, this AR technology may provide the surgeon an opportunityto practice the surgery before performing on the patient. For example,by leveraging the medical images taken of the patient, AR-content may begenerated for creating or simulation the unique anatomy of the patient,and the user may be able to perform the surgery on a mannequin, a humanpatient simulator, through visualization, and/or the like. In addition,even where the surgery is being simulated, markers—e.g., correspondingto the markers 192—may be placed on a human person simulator or othersimulated object, and the AR-content may be generated accurately withlocalization corresponding to the actual patient. As a result, theincisions, locations of vital structures, and/or other informationcorresponding to the surgery or procedure may be accurately displayedeven during the simulation—thereby increasing the accuracy of thesimulation and leading to more comfort and precision during the actualsurgery or procedure. In some embodiments, the AR technology may be usedfor general diagnosis and examination purposes. For example, imaging canbe overlaid from a patient using AR to aid in the diagnosis process toidentify or rule out abnormal vs. normal findings.

With respect to FIG. 1E, in some embodiments, one or more loupe(s) 122may be used by the user 160 of the head-worn device 130. The loupe(s)122 may have any magnification depending on the preferences of the user160 and/or the type of procedure being performed. For example, theloupe(s) 122 may have a magnification of 2.5×, 3.5×, 4×, 4.5×, etc. Insome examples, the image data captured by the camera(s) 116 may becaptured at a zoom or aspect ratio that mimics the magnification of theloupe(s) 122—whether or not the loupe(s) 122 are actually used (e.g.,where loupe(s) 122 aren't used, the image data may still be generatedwith a field of view that mimics the use of loupe(s) 122). For example,when capturing image data of a medical procedure for streaming and/orlater viewing, the viewer (e.g., a student, physician, interested party,etc.) may desire to view the procedure from the viewpoint of the user160 of the head-worn device 130. As such, the zoom of the camera(s) 116may be adjusted such that the field of view captured in the image datais similar to the field of view of the user 160 through the loupe(s)122. In some embodiments, the zoom may be optical zoom, while in others,the zoom may be digital zoom. In addition, in some examples, the imagequality or aspect ratio may be such that cropping the images may resultin a field of view that mimics the field of view of the user 160 throughthe loupe(s) 122 without any digital or optical zoom. For example, where4K image data is generated, the images may be cropped to 1080p qualityat an aspect ratio or field of view that mimics that created by theloupe(s) 122. In such examples, any number of different crops may bemade to adjust the field of view to mimic views of loupe(s) 122 ofvarying magnifications. For example, a first user may wish to view theimage data (e.g., video of a procedure) at a first magnificationmimicking 2.5× loupe(s) 122, while a second user may wish to view theimage data at a second magnification mimicking 3.5× loupe(s) 122. Assuch, different crops may be generated to accommodate both users. Inother examples, such as where two or more cameras 116 are used,different cameras 116 may capture image data at varying zooms, aspectratios, and/or qualities to provide different fields of view,magnifications, and/or other parameters for viewing by users.

The I/O device(s) 126 may include a microphone(s), a speakers(s), atouch surface(s), a button(s), a switch(es), and/or other device types.For example, the microphone(s) may be used to provide voice commands, togenerate recordings for transcriptions or later review, to providecommentary during surgery, to capture commentary during procedures forlistening while viewing the images and/or video generated by thecamera(s) 116, etc. Where voice commands are used, the voice commandsmay be used to activate the camera(s) 116, deactivate the camera(s) 116,start a recording, pause a recording, resume a recording, activate thelight(s) 124, deactivate the light(s) 124, and/or other perform otheroperations or tasks. The speaker(s) may be used to provide feedback tothe user 160, such as to capture certain images, to make certainadjustments, etc.

The power component 164 may include one or more of the battery(ies) 114,one or more components of the AR system 120, the processor(s) 118, themicrophone(s), the speaker(s), the button(s), the switches(s), other I/Odevice(s) 126, or the communication interface 128. In some embodiments,the head-worn device 130 may include one or more of the IMU(s) 112,camera(s) 116, light(s) 124, loupe(s) 122, one or more components of theAR system 120, the processor(s) 118, the microphone(s), the speaker(s),the button(s), the switches(s), other I/O device(s) 126, the motor(s)182, the heat sensor(s) 180, the battery(ies) 114, or the communicationinterface 128.

The electrical coupling 166 may electrically couple the head-worn device130 to the power component 164. The electrical coupling may include ahard-wired connection via one or more electrical cables. In someembodiments, the electrical coupling may include ribbon wire that may becapable of efficiently transporting or transmitting image data, audiodata, AR data, and/or electrical power (e.g., from the battery(ies) 114to one or more components of the head-worn device 130). Although theelectrical coupling is illustrated herein, in some embodiments, thehead-worn device 130 may communicate data wirelessly to the powercomponent 164, and/or the power component 164 may not be implemented. Insuch examples, the head-worn device 130 may include a battery(ies) 114disposed thereon for powering one or more components. For example, thehead-worn device 130 may generate audio, image, and/or other data types,and may transmit the data to the power component 164 (where present),the local client device(s) 104, and/or the server(s) 106. For example, ahousing may be disposed on the head-worn device 130 that may include abattery(ies) 114 and/or one or more components of the head-worn device130—such as the camera(s) 116, light(s) 124, etc. In some embodiments,the power component 164 may, receive wirelessly and/or via a wiredconnection, the audio, image, and/or other data from the head-worndevice, process the data (e.g., analyze, compress, encode, etc.), andtransmit the data to the local client device(s) 104 and/or the server(s)106.

The remote client device(s) 108 may include one or more clientapplication(s) 150 for accessing, replaying, or storing image data,audio data, medical imaging data, and/or other data from one or moreprocedures and/or from patient data (e.g., via a patient portal). Forexample, in some embodiments, the server(s) 106 may stream or otherwisetransmit (e.g., for storage) image data, audio data, and/or other datato the remote client device(s) 108 via streaming application(s) 140. Insuch examples, prior to sending any patient data that may be protectedunder medical rules and regulation, such as the Health InsurancePortability and Accountability Act (HIPAA), an authenticator 146 mayauthenticate the remote client device(s) 108—or the user thereof. Forexample, credential of the user within the client application(s) 150 maybe verified, a token may be required, and/or other authenticationprocesses may be implemented. In some examples, the users of the remoteclient device(s) 108 may be students, patients, doctors, and/or othertypes of persons, and the authentication steps may vary depending on thetype of user in order to comply with medical rules and regulations. Theclient application(s) 150 are described in more detail herein at leastwith respect to FIGS. 3A-3G. Processes or operations of the clientapplication(s) 150 may be executed by the processor(s) 154.

In some non-limiting embodiments, the loupe system(s) 102 may include aprocessor(s) 118 and/or one or more applications or instructions thereonfor performing the analysis to determine the presence of the personand/or a feature(s) thereof, to determine and/or project AR-content,and/or to perform any other processes or operations described herein.The processor(s) 118 may be a component of the head-worn device 130, thepower component 164, or a combination thereof. As such, the processingof the computer vision, facial recognition, and/or other detectionalgorithms may be executed by a processor(s) 118 of the head-worn device130, the power component 164, or a combination thereof.

In other embodiments, a local client device(s) 104 may include one ormore applications (e.g., medical application(s) 134) or instructionsthereon for performing the analysis to determine the presence of theperson and/or a feature(s) thereof, to determine and/or generateAR-content, and/or to perform any other processes or operationsdescribed herein. For example, one or more processors 132 of the localclient device(s) 104 may receive the image data from the loupe system(s)102, analyze the image data using a computer vision, facial recognition,and/or other detection algorithm, and determine whether or not a person168 or a feature thereof is present. Once a determination is made that aperson 168—or a feature thereof—is present, a signal may be generated bythe local client device(s) 104 and transmitted to the loupe system(s)102 (e.g., to the power component 164 and/or a component of thehead-worn device 130) to cause the light(s) 124 to be activated.Similarly, for AR-content generation, the local client device(s) 104 mayreceive the image data, medical imaging data, and/or other data typesthat may be used to generate and/or render the AR-content.

In still further embodiments, the server(s) 106 may include one or moreapplications (e.g., medical application(s) 134, streaming application(s)140, etc.) or instructions thereon for performing the analysis todetermine the presence of the person and/or a feature(s) thereof, todetermine and/or generate AR-content, and/or to perform any otherprocesses or operations described herein. For example, one or moreprocessors 144 of the server(s) 106 may receive the image data from theloupe system(s) 102 and/or the local client device(s) 104, analyze theimage data using a computer vision, facial recognition, and/or otherdetection algorithm, and determine whether or not a person 168 or afeature thereof is present. Once a determination is made that a person168—or a feature thereof—is present, a signal may be generated by theserver(s) 106 and transmitted to the loupe system(s) 102 (e.g., to thepower component 164 and/or a component of the head-worn device 130), viathe local client device(s) 104, in embodiments, to cause the light(s)124 to be activated. Similarly, for AR-content generation, the server(s)106 may receive the image data, medical imaging data, and/or other datatypes that may be used to generate and/or render the AR-content.

In some embodiments, the server(s) 106 may render the AR-content in acloud environment, capture the rendering in display or projection data,and transmit the display or projection data to the loupe system(s) 102(e.g., via the local client device(s) 104) for display or projection asAR-content. As such, the analysis, generation, and rendering of theAR-content may be performed by the server(s) 106—thereby reducing thecomputation expense of the loupe system(s) 102 and/or the local clientdevice(s) 104—and the loupe system(s) 102 may display the alreadyrendered and captured display data. Using this approach, the computationresources and efficiency of a cloud-based platform—which may include acloud-based GPU infrastructure—may be leveraged to generate higherquality display data for the AR system 120 with lower latency, andwithout requiring the local client device(s) 104 to include sufficientstorage space and/or processing power.

The image data, AR-content, audio data, and/or other data types may becommunicated between the loupe system(s) 102 and the local clientdevice(s) 104 using a communication interface 128 of the loupe system(s)102 and a communication interface 136 of the local client device(s) 104.Similarly, signals indicative of activation or deactivation of the lightmay be transmitted between the local client device(s) 104 and the loupesystem(s) 102 via the respective communication interfaces. For example,the loupe system(s) 102 and the local client device(s) 104 maycommunicate over one or more networks 110, such as a local area network(LAN), a wide-area network (WAN), an ultra-low power network (ULP), alow-power network, a short range network, a low power wide-area network(LPWAN), and/or another network type. In some examples, the loupesystem(s) 102 and the client device(s) 104 may communicate over one ormore of Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, ZigBee, WiMax,EnOcean, Eddystone, adaptive network topology (ANT), near-fieldcommunication (NFC), IEEE 802.15, ISA100.11a, DigiMesh, WirelessHART,Ethernet, cellular (e.g., 2G, 3G, 4G, LTE, 5G, etc.), LoRaWAN, NB-IoT,SigFox, Weightless, Z-wave, and/or another communication protocol(s).

Similarly, in some embodiments, the image data, AR-content, audio data,and/or other data types may be communicated between the loupe system(s)102, the local client device(s) 104, and/or server(s) 106 using acommunication interface 128 of the loupe system(s) 102, a communicationinterface 136 of the local client device(s) 104, and/or a communicationinterface 142 of the server(s) 106. As a non-limiting example, imagedata may be generated by the loupe system(s) 102, transmitted to a localclient device(s) 104, and the local client device(s) 104 may transmitthe image data to the server(s) 106 for analysis. Similarly, thedeterminations, or signals representative thereof, that the light(s) 124should be turned on or off may be communicated to the loupe system(s)102 by the server(s) 106 (via the local client device(s) 104, inembodiments). For example, the loupe system(s) 102 and the local clientdevice(s) 104 may communicate over one or more networks 110, such asthose described herein, and the loupe system(s) 102 and/or the clientdevice(s) 104 may communicate with the server(s) 106 over one or more ofa local area network (LAN) (e.g., Wi-Fi, Ethernet, etc.), a wide-areanetwork (WAN) (e.g., the Internet, a public switched telephone networkPSTN), etc.), an ultra-low power network (ULP), a low-power network, ashort range network, a low power wide-area network (LPWAN), and/oranother network type. As such, the image data, the activation signals,the deactivation signals, and/or other communications may be transmittedbetween and among the loupe system(s) 102, the local client device(s)104, and/or the server(s) 106.

Similarly, the server(s) 106, the loupe system(s) 102, and/or the localclient device(s) 104 may similarly communicate with the remote clientdevice(s) 108 over the network(s) 110—including but not limited to thosedescribed herein—via communication interface 152 of the remote clientdevice(s) 108. The local client device(s) 104 may, without limitation,be co-located with the loupe system(s) 102 such that a LAN may be used,or a direct wired connection may be used to transmit data between andamong the devices. The remote client device(s) 108 may be at a remotelocation with respect to the loupe system(s) 102, the local clientdevice(s) 104, and/or the server(s) 106. For an example, the loupesystem(s) 102 and the local client device(s) 104 may be located at amedical facility, the server(s) 106 may be located at the medicalfacility and/or at a remote cloud-based facility, and the remote clientdevice(s) 108 may be located with one or more users at any location—suchas a home of the user, an educational environment, etc.

Now referring to FIG. 2A, FIG. 2A is an example illustration ofeye-glasses electrically coupled to a power component in a loupe system,in accordance with some embodiments of the present disclosure. Theeye-glasses represent one possible embodiment of the head-worn device130 in the loupe system(s) 102 described herein. FIG. 2A providesanother example illustration of the loupe system(s) 102 from a differentperspective for illustrative purposes only. In some embodiments, thecamera(s) 116 may be positioned as illustrated in FIG. 2A, while inother embodiments, the camera(s) 116 may be positioned differently. Forexample, the camera(s) 116 may all be in a single housing centrallypositioned—or otherwise positioned—on the head-worn device 130. Eachcamera 116, where two or more are present, may include a different fieldof view, a different type (e.g., wide view, stereo, 360 degree, etc.), adifferent location and/or orientation, a different aspect ratio, adifferent image quality, and/or other different parameters.

Now referring to FIG. 2B, FIG. 2B is an example illustration of ahousing associated with a head-worn device within a loupe system, inaccordance with some embodiments of the present disclosure. The housingmay include a camera housing, such as a housing for two or more cameras116 (e.g., 4K cameras 116D and first-person view (FPV) camera 116E).Additional components of the housing may include a mode LED 260 toindicate a current operating mode (e.g., on/off, recording, notrecording, charged, low-battery, etc.), an antenna 258 for receiving andtransmitting, a battery connector 256 for connecting power to componentsof the device, a ground 254, a 5-12 V connection 252, a ground 250,video 248, menu 246, ground 244, a 5-12 V connection 242, a ground 240,video 238, menu 236, ground 234, receiver 232 for receiving data,transmitter 230 for transmitting data, a heat sink 228 for dispersingheat from components of the device, a Wi-Fi/recode button 226 forconnecting to other devices (e.g., the local client device(s) 104), apower/mode button 224 for turning the device on/off, a Wi-Fi LED 220 toindicate a Wi-Fi connection (or other type of connection) is establishedor active, and a memory card 222 (e.g., a TransFlash (TF) card, a microSD card, etc.) for storing data that is received (e.g., AR-content,audio data, etc.) and/or transmitted (e.g., audio data, image data,input data, state data, etc.).

Now referring to FIGS. 3A-3G, FIGS. 3A-3G are example illustrations ofgraphical user interfaces (GUIs) for streaming or viewing medicalprocedures and/or viewing patient information within one or moreapplications, in accordance with some embodiments of the presentdisclosure. The one or more applications may include the clientapplication(s) 150 of the remote client device(s) 108 of FIG. 1E, andthe streams of data and/or access to stored data may be received and/oraccessed via the streaming application(s) 140 on the server(s) 106—e.g.,after authentication by the authenticator 146. The client application(s)150 may be used to view videos, images, and/or other recordings ofprocedures, surgeries, and/or the like, that are generated using theloupe system(s) 102. For example, as a user 160 (e.g., doctor, dentist,surgeon, nurse, etc.) of the loupe system(s) 102 performs an operation,procedure, general exam, etc., the user 160 may desire to record, store,and/or stream the recording of the procedure for viewing by one or moreusers on a remote client device(s) 108 (e.g., a laptop, computer,tablet, phone, etc.). This information may be used for educationalpurposes, in some embodiments. As described herein, the recordings maybe generated by the one or more camera(s) 116 at one or more fields ofview, magnifications or zooms, aspect rations, etc., such that one ormore perspectives of the operation, procedure, etc. are available forviewing. In some embodiments, the client application(s) 150 may includea patient portal for patients to access medical information, such asrecordings of procedures, medical imaging data, insurance information,images of cavities, tooth replacements, etc., and/or other informationabout the patient for their own use or safekeeping. In any example, theGUI(s) described with respect to FIGS. 3A-3G may include varyingexamples for streaming recordings, viewing stored recordings (e.g., onthe remote client device(s), on the server(s) 106, etc.), searching forrecordings or live-streams, changing settings, viewing patientinformation, viewing medical imaging data, etc.

FIG. 3A includes a GUI 302 for viewing pre-recorded or live-streamedvideos. For example, the user of the remote client device(s) 108 mayhave searched for certain video of certain procedure types, surgerytypes, etc., within GUI 310, and received the list of videos 304 (e.g.including videos 304A, 304B, and 304C). FIG. 3B includes the GUI 310 forsearching for videos, procedures, doctors or other practitioners, etc.For example, a user may enter a search into search bar 312 using digitalkeypad 314 to retrieve current live streams, future live streams,pre-recorded videos, etc. In any example, the user may desire to savethe videos or other data—where authorized—and may do so on his or herremote client device(s) 108. Once a video 322 or other recording type isselected, GUI 320 may playback the video 322. One or more controlelements 324 may be available within the GUI 320 for controlling theplayback (e.g., play, stop, pause, record, fast-forward, rewind, capturescreenshot, add notes, add comments, provide commentary, etc.). Theclient application(s) 150 may include various settings 332, and GUI 330may allow the user to adjust settings 332 (e.g., settings 332A, 332B,and 332F). For example, the user may be able to adjust the imagequality, frame rate, other image or video settings, zoom level,magnification level, video types, login information, account type, audiosettings, credential or authentication information, and/or the like.

GUI 340 may represent a GUI of a patient portal where a patient or usermay access health information. For example, an image 342 of the patient,patient personal information 344, and/or patient medical information 346may be accessible and/or editable within the GUI 340. The GUI 350 mayallow the patient to access videos 350 (e.g., videos 352B, 352D, and352F) of surgeries that were performed on them, that were performed onothers, that are similar to the surgery they may be undergoing, thathave been performed by their doctor, etc. Similarly, GUI 360 may allowthe patient to access medical imaging data 362 (e.g., medical imagingdata 362A, 362B, and 362C). In addition, as described herein, the GUI(s)may be accessed to view insurance information, images taken of cavities,etc., and/or to access other patient information—such as results oftests, etc.

Now referring to FIGS. 4-6, each block of methods 400, 500, and 600,described herein, comprises a computing process that may be performedusing any combination of hardware, firmware, and/or software. Forinstance, various functions may be carried out by a processor executinginstructions stored in memory. The methods may also be embodied ascomputer-usable instructions stored on computer storage media. Themethods may be provided by a standalone application, a service or hostedservice (standalone or in combination with another hosted service), or aplug-in to another product, to name a few. In addition, methods 400,500, and 600 are described, by way of example, with respect to the loupesystem(s) 102 of FIGS. 1A-1E. However, these methods may additionally oralternatively be executed by any one system, or any combination ofsystems, including, but not limited to, those described herein.

Now referring to FIG. 4, FIG. 4 is a flow diagram for an example method400 of using facial recognition to activate or deactivate a light sourceof eye-glasses in a loupe system, in accordance with some embodiments ofthe present disclosure. The method 400, at block B402, includesgenerating, using a camera disposed on a head-mounted device, image datarepresentative of one or more images in a field of view of the camera.For example, the camera(s) 116 may generate image data representative ofa field(s) of view of the camera(s) 116.

The method 400, at block B404, includes wirelessly transmitting theimage data to a client device. For example, the image data may bewirelessly transmitted to the local client device(s) 104 over one ormore networks 110.

The method 400, at block B406, includes wirelessly receiving, from theclient device and based at least in part on a determination that aperson is depicted in the image data, a signal. For example, the localclient device(s) 104 may analyze the image data using one or more facialrecognition, computer vision, and/or other detection algorithms todetermine whether a person—or a face or other feature thereof—isdepicted in the image data. Once a determination is made that a personis present, a signal may be transmitted to the loupe system(s) 102—suchas the head-worn device 130, via the power component 164, inembodiments—that is representative of an activation request foractivating the light(s) 124.

The method 400, at block B408, includes, based at least in part on thesignal, activating a light source disposed on the head-mounted device.For example, the signal may cause the light(s) 124 to be activated.

Now referring to FIG. 5, FIG. 5 is a flow diagram for an example method500 of streaming a medical procedure captured using a head-mountedcamera to one or more remote client devices, in accordance with someembodiments of the present disclosure. The method 500, at block B502,includes receiving, from a client device, image data generated by acamera disposed on eye-glasses of a loupe system communicatively coupledto the client device. For example, the server(s) 106 may receive, fromthe local client device(s) 104, image data generated by a camera(s) 116of the loupe system(s) 102.

The method 500, at block B504, includes determining one or more remoteclient devices authorized for viewing a medical procedure. For example,the authenticator 146 of the server(s) 106 may determine which of theremote client device(s) 108 are authorized for viewing—either via a livestream, a pre-recorded stream, or as downloadable content—the medicalprocedure.

The method 500, at block B506, includes encoding the image data togenerate encoded image data. For example, the image data may be encodedinto a format for streaming to the remote client device(s) 108, such asinto a inter frame only format (e.g., P-frame only), or another suitableformat for streaming.

The method 500, at block B508, includes streaming the encoded image datato the one or more remote client devices. For example, the image data,after encoding and/or compression, may be streamed to one or more of theremote client device(s) 108 for viewing, storage, or otherwise.

Now referring to FIG. 6, FIG. 6 is a flow diagram for an example method600 of using AR functionality to provide virtual visualizations toassist in medical procedures, in accordance with some embodiments of thepresent disclosure. The method 600, at block B602, includes receivingmedical imaging data generated by an imaging device, the medical imagingdata representative of a person having one or more markers attachedthereto. For example, medical imaging data may be received, where themedical imaging data was generated by one or more medical imagingdevices and of a person 168 having one or more markers 172 attachedthereto. In other examples, in addition to or alternatively from markers172, one or more features of the person may be represented in themedical imaging data—such as joints, eyes, ears, etc.

The method 600, at block B604, includes receiving image data generatedby at least one camera disposed on AR glasses, the image datarepresentative of fields of view of the at least one camera, the fieldsof view including a person with one or more markers attached thereto.For example, image data may be received from the camera(s) 116 of theloupe system(s) 102, where the image data is representative of theperson 168 having one or more markers 172 attached thereto (and/orrepresentative of one or more features of the person 168).

The method 600, at block B606, includes aligning the image data with themedical imaging data to generate projection data based at least in parton representations of the one or more markers in the same. For example,the image data and the medical imaging data may be aligned—e.g., usingthe marker(s) 172 and/or features of the person 168—such that AR-contentmay be generated for projection at a particular location in theenvironment and at an accurate scale. In some embodiments,triangulation, 2D mapping, 3D mapping, or other techniques may be usedto determine the alignment of the image data and the medical imagingdata for generating the visualizations within the AR-content.

The method 600, at block B608, includes transmitting the projection datato cause the AR glasses to project a first representation of theprojection data on at least one lens of the AR glasses such that atleast a portion of a second representation of the medical imaging datais virtually overlaid on the person. For example, the projection datamay be transmitted to the loupe system(s) 102—e.g., to the AR system 120of the head-worn device 130—to cause the loupe system(s) 102 to projectthe AR-content onto the lens(es) of the head-worn device 130 such that arepresentation of the medical imaging data is virtually overlaid in ARon the person 168.

Example Computing Device

FIG. 7 is a block diagram of an example computing device 700 suitablefor use in implementing some embodiments of the present disclosure. Forexample, some or all of the components of the computing device 700 maybe included or used in the loupe system(s) 102, the local clientdevice(s) 104, the server(s) 106, and/or the remote client device(s)108, in addition to or alternatively from the other components orfeatures described herein. The computing device 700 may include a bus702 that directly or indirectly couples the following devices: memory704, one or more central processing units (CPUs) 706, one or moregraphics processing units (GPUs) 708, a communication interface 710,input/output (I/O) ports 712, input/output components 714, a powersupply 716, and one or more presentation components 718 (e.g.,display(s)).

Although the various blocks of FIG. 7 are shown as connected via the bus702 with lines, this is not intended to be limiting and is for clarityonly. For example, in some embodiments, a presentation component 718,such as a display device, may be considered an I/O component 714 (e.g.,if the display is a touch screen). As another example, the CPUs 706and/or GPUs 708 may include memory (e.g., the memory 704 may berepresentative of a storage device in addition to the memory of the GPUs708, the CPUs 706, and/or other components). In other words, thecomputing device of FIG. 7 is merely illustrative. Distinction is notmade between such categories as “workstation,” “server,” “laptop,”“desktop,” “tablet,” “client device,” “mobile device,” “handhelddevice,” “game console,” “electronic control unit (ECU),” “virtualreality system,” and/or other device or system types, as all arecontemplated within the scope of the computing device of FIG. 7.

The bus 702 may represent one or more busses, such as an address bus, adata bus, a control bus, or a combination thereof. The bus 702 mayinclude one or more bus types, such as an industry standard architecture(ISA) bus, an extended industry standard architecture (EISA) bus, avideo electronics standards association (VESA) bus, a peripheralcomponent interconnect (PCI) bus, a peripheral component interconnectexpress (PCIe) bus, and/or another type of bus.

The memory 704 may include any of a variety of computer-readable media.The computer-readable media may be any available media that may beaccessed by the computing device 700. The computer-readable media mayinclude both volatile and nonvolatile media, and removable andnon-removable media. By way of example, and not limitation, thecomputer-readable media may comprise computer-storage media andcommunication media.

The computer-storage media may include both volatile and nonvolatilemedia and/or removable and non-removable media implemented in any methodor technology for storage of information such as computer-readableinstructions, data structures, program modules, and/or other data types.For example, the memory 704 may store computer-readable instructions(e.g., that represent a program(s) and/or a program element(s), such asan operating system. Computer-storage media may include, but is notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical disk storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or any other medium which may be used to storethe desired information and which may be accessed by computing device700. As used herein, computer storage media does not comprise signalsper se.

The communication media may embody computer-readable instructions, datastructures, program modules, and/or other data types in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” mayrefer to a signal that has one or more of its characteristics set orchanged in such a manner as to encode information in the signal. By wayof example, and not limitation, the communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, RF, infrared and other wireless media.Combinations of any of the above should also be included within thescope of computer-readable media.

The CPU(s) 706 may be configured to execute the computer-readableinstructions to control one or more components of the computing device700 to perform one or more of the methods and/or processes describedherein. The CPU(s) 706 may each include one or more cores (e.g., one,two, four, eight, twenty-eight, seventy-two, etc.) that are capable ofhandling a multitude of software threads simultaneously. The CPU(s) 706may include any type of processor, and may include different types ofprocessors depending on the type of computing device 700 implemented(e.g., processors with fewer cores for mobile devices and processorswith more cores for servers). For example, depending on the type ofcomputing device 700, the processor may be an ARM processor implementedusing Reduced Instruction Set Computing (RISC) or an x86 processorimplemented using Complex Instruction Set Computing (CISC). Thecomputing device 700 may include one or more CPUs 706 in addition to oneor more microprocessors or supplementary co-processors, such as mathco-processors.

The GPU(s) 708 may be used by the computing device 700 to rendergraphics (e.g., 3D graphics). The GPU(s) 708 may include hundreds orthousands of cores that are capable of handling hundreds or thousands ofsoftware threads simultaneously. The GPU(s) 708 may generate pixel datafor output images in response to rendering commands (e.g., renderingcommands from the CPU(s) 706 received via a host interface). The GPU(s)708 may include graphics memory, such as display memory, for storingpixel data. The display memory may be included as part of the memory704. The GPU(s) 708 may include two or more GPUs operating in parallel(e.g., via a link). When combined together, each GPU 708 may generatepixel data for different portions of an output image or for differentoutput images (e.g., a first GPU for a first image and a second GPU fora second image). Each GPU may include its own memory, or may sharememory with other GPUs. In some examples, the GPU(s) 708 may rendergraphics using various technologies, such as ray-tracing technologies,such as to generate the AR-content for display using the loupe system(s)102.

In examples where the computing device 700 does not include the GPU(s)708, the CPU(s) 706 may be used to render graphics.

The communication interface 710 may include one or more receivers,transmitters, and/or transceivers that enable the computing device 700to communicate with other computing devices via an electroniccommunication network, included wired and/or wireless communications.The communication interface 710 may include components and functionalityto enable communication over any of a number of different networks, suchas wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE,ZigBee, etc.), wired networks (e.g., communicating over Ethernet),low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and/or theInternet.

The I/O ports 712 may enable the computing device 700 to be logicallycoupled to other devices including the I/O components 714, thepresentation component(s) 718, and/or other components, some of whichmay be built in to (e.g., integrated in) the computing device 700.Illustrative I/O components 714 include a microphone, mouse, keyboard,joystick, game pad, game controller, satellite dish, scanner, printer,wireless device, etc. The I/O components 714 may provide a natural userinterface (NUI) that processes air gestures, voice, or otherphysiological inputs generated by a user. In some instances, inputs maybe transmitted to an appropriate network element for further processing.An NUI may implement any combination of speech recognition, stylusrecognition, facial recognition, biometric recognition, gesturerecognition both on screen and adjacent to the screen, air gestures,head and eye tracking, and touch recognition (as described in moredetail below) associated with a display of the computing device 700. Thecomputing device 700 may be include depth cameras, such as stereoscopiccamera systems, infrared camera systems, RGB camera systems, touchscreentechnology, and combinations of these, for gesture detection andrecognition. Additionally, the computing device 700 may includeaccelerometers or gyroscopes (e.g., as part of an inertia measurementunit (IMU)) that enable detection of motion. In some examples, theoutput of the accelerometers or gyroscopes may be used by the computingdevice 700 to render immersive augmented reality or virtual reality.

The power supply 716 may include a hard-wired power supply, a batterypower supply, or a combination thereof. The power supply 716 may providepower to the computing device 700 to enable the components of thecomputing device 700 to operate.

The presentation component(s) 718 may include a display (e.g., amonitor, a touch screen, a television screen, a heads-up-display (HUD),other display types, or a combination thereof), speakers, and/or otherpresentation components. The presentation component(s) 718 may receivedata from other components (e.g., the GPU(s) 708, the CPU(s) 706, etc.),and output the data (e.g., as an image, video, sound, etc.).

The disclosure may be described in the general context of computer codeor machine-useable instructions, including computer-executableinstructions such as program modules, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program modules including routines, programs,objects, components, data structures, etc., refer to code that performparticular tasks or implement particular abstract data types. Thedisclosure may be practiced in a variety of system configurations,including handheld devices, consumer electronics, general-purposecomputers, more specialty computing devices, etc. The disclosure mayalso be practiced in distributed computing environments where tasks areperformed by remote-processing devices that are linked through acommunications network.

As used herein, a recitation of “and/or” with respect to two or moreelements should be interpreted to mean only one element, or acombination of elements. For example, “element A, element B, and/orelement C” may include only element A, only element B, only element C,element A and element B, element A and element C, element B and elementC, or elements A, B, and C. In addition, “at least one of element A orelement B” may include at least one of element A, at least one ofelement B, or at least one of element A and at least one of element B.Further, “at least one of element A and element B” may include at leastone of element A, at least one of element B, or at least one of elementA and at least one of element B.

The subject matter of the present disclosure is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of thisdisclosure. Rather, the inventors have contemplated that the claimedsubject matter might also be embodied in other ways, to includedifferent steps or combinations of steps similar to the ones describedin this document, in conjunction with other present or futuretechnologies. Moreover, although the terms “step” and/or “block” may beused herein to connote different elements of methods employed, the termsshould not be interpreted as implying any particular order among orbetween various steps herein disclosed unless and except when the orderof individual steps is explicitly described.

What is claimed is:
 1. A method comprising: generating, using a cameradisposed on a head-mounted device, image data representative of one ormore images in a field of view of the camera; wirelessly transmittingthe image data to a client device; wirelessly receiving, from the clientdevice and based at least in part on a determination that a person isdepicted in the image data, a signal; based at least in part on thesignal, activating a light source disposed on the head-mounted device.2. The method of claim 1, wherein the head-mounted device includeseye-glasses coupled to a battery.
 3. The method of claim 2, wherein theeye-glasses are coupled to the battery using ribbon wire.
 4. The methodof claim 1, wherein the determination that a person is depicted in theimage data is by at least one of the client device or a server, and thedetermination is based on analyzing the image data using a facialrecognition algorithm.
 5. The method of claim 1, wherein the camera iselectrically coupled to a housing discrete from the eye-glasses, thehousing including a battery, a wireless communication interface, and atleast one component associated with processing the image data.
 6. Themethod of claim 1, wherein the eye-glasses further include one or moreinertial measurement unit (IMU) sensors disposed thereon, the one ormore IMU sensors are zeroed to a zeroed position based at least in parton the signal, and the method further comprises: tracking sensor datagenerated by the one or more IMU sensors; estimating an amount ofmovement of the eye-glasses relative to the zeroed position based atleast in part on the sensor data; and when the amount of movement isabove a threshold, deactivating the light source, wherein the thresholdis one of a default threshold or a threshold corresponding to aparticular patient.
 7. The method of claim 6, wherein, when a subsequentamount of movement indicates that the IMU sensors are within anotherthreshold of the zeroed position, reactivating the light source.
 8. Themethod of claim 7, wherein: the image data is generated and transmittedcontinuously; the generating and the transmitting of the image data issuspended based at least in part on the one or more sensors beingzeroed; and when the amount of movement is above the threshold, thegenerating and the transmitting of the image data is resumed.
 9. Themethod of claim 8, further comprising: receiving a second signal fromthe client device and based at least in part on another determinationthat the person is depicted in subsequent image data generated afterresuming the generating and the transmitting of the image data; andreactivating the light source.
 10. The method of claim 1, wherein theeye-glasses are part of a loupe system, and the eye-glasses furtherinclude at least one loupe disposed thereon.
 11. The method of claim 1,wherein the image data is stored on a server for retrieval by one ormore remote client devices.
 12. The method of claim 11, wherein thestored image data is retrievable via an application on the one or moreremote client devices, the application for educational purposes inviewing medical procedures from a perspective of a user of theeye-glasses.
 13. A method comprising: receiving, from a client device,image data generated by a camera disposed on eye-glasses of a loupesystem communicatively coupled to the client device, the image datagenerated by the camera during at least a portion of a medicalprocedure; determining one or more remote client devices authorized forviewing the medical procedure, the one or more client devices authorizedbased at least in part on an authentication procedure within respectiveinstantiations of an application executing on the one more remote clientdevices; encoding the image data to generate encoded image data; andstreaming the encoded image data to the one or more remote clientdevices to cause display of the encoded image data on respectivedisplays of the one or more remote client devices.
 14. The method ofclaim 13, wherein: the eye-glasses include one or more loupes disposedthereon; the one or more loupes having a magnification factor; and acamera zoom is determined to correspond to the magnification factor. 15.The method of claim 13, wherein the client device is wirelesslycommunicatively coupled to the loupe system locally, and the server andthe one or more remote client devices are remotely located with respectto both the client device and the loupe system.
 16. The method of claim13, wherein the image data is further analyzed to determine whether aface of a person is present, and when the face of the person isdetected, generating and transmitting a signal to the loupe system tocause a light disposed on the eye-glasses to activate a light sourcedisposed thereon.
 17. A loupe system comprising: eye-glasses including:one or more cameras disposed thereon to generate image data; and a lightsource disposed thereon; a power device electrically coupled to theeye-glasses, the power device including: a battery to power at least theone or more cameras and the light source; an encoder for generatingencoded image data using the image data; a wireless communicationcomponent to transmit the encoded image data; an application executingon a client device communicatively coupled to the power device, theapplication to: determine, based at least in part on the encoded imagedata, that a face of a person is represented by the encoded image data;and generate and transmit, via another wireless communication componentof the client device, a signal to the power device, the signal causingthe light source to be activated.
 18. The loupe system of claim 17,wherein the eye-glasses further include one or more inertial measurementunit (IMU) sensors that are zeroed to a zeroed position based at leastin part on the signal being received by the power device.
 19. The loupesystem of claim 18, wherein: sensor data generated by the one or moreIMU sensors is tracked using one or more processors of the power device;an amount of movement of the eye-glasses relative to the zeroed positionis determined using the one or more processors and based at least inpart on the sensor data; and when the amount of movement is above athreshold, the light source is deactivated.
 20. The loupe system ofclaim 17, wherein the eye-glasses further include at least one loupedisposed thereon.